The term “multimedia content” refers to a wide variety of material such as motion picture video, television programming, audiovisual presentations, web pages and the other types of content. Multimedia content can be continuous (e.g., video or audio) or non-continuous (e.g., text or html for a web page). In some examples, the multimedia content is time-based, meaning that portions of the content are to be presented in a particular order in time and may include different elements that need to be synchronized relative to other elements of the content. For example, motion picture or video content presents images in a specific order and includes both video and audio elements that are synchronized for accurate playback. Additionally, motion picture or television programming may include subtitles or a second audio channel in another language that would need to be synchronized with the video element for accurate playback.
The various elements in such content are synchronized by referencing a common system clock. This time reference is generally associated with each of the elements and is used to ensure that the presentation of all the content elements, or single components of content, is synchronized at playback. In a particular content delivery system, there may be a number of timestamps used to ensure accurate reproduction of the content on a playback device.
This synchronization of various content elements is complicated when content is streamed over a distribution network such as a cable or satellite system, particularly where the playback speed may be controlled by an end user. For example, in a system with video-on-demand capability, a particular movie may be streamed from a headend over a cable or satellite system to local content servers. The local content servers may then stream the content to specific requesting locations “on demand.” This may also include allowing the recipient or requesting user to pause, rewind or fast forward the content during playback, i.e., trick play.
In order for a content source or other device to stream multimedia content over a communication path, such as a network or a satellite link, the elements of the content are broken into presentation units, each with associated timestamps from the common system clock. A presentation unit is defined in the Motion Picture Experts Group 2, Part 1, standard. For example, in video, a presentation unit includes all the coded data for one video frame or one video field. Once the content is broken into presentation units, the resulting presentation units are encapsulated in one or more data packets for transmission. Typically, these packets are interleaved into a continuous packetized stream such that synchronized content elements for multiple programs are delivered to various receiving devices simultaneously, all the elements of the content arriving in a timely fashion. Data packets differ depending on the communication medium over which data content is transmitted.
In addition to the synchronization of elements described above, data must continuously arrive at the recipient device within a limited window of time or the playback is interrupted. Because multimedia data content may include massive amounts of data, streaming systems cannot always rely on buffering data in the receiving device to maintain the data stream. This may also complicate the need to synchronize the arriving content elements.
Existing content servers, such as equipment that delivers time-based content to a receiving device, implement timestamp updates by searching the component streams for the relevant timestamp and updating or adding a new timestamp as the content is streamed out. In many instances, the individual content elements are separated, updated with new timestamps and recombined to form a new packetized stream with the correct timing. This process must be performed in real-time for each active stream in the system and reoccurs for each request for the stream. In some instances, this processing can fail to maintain valid data streams, which results in poor presentation on the recipient's playback device.
If navigation or trick play is supported, the content server will typically build or obtain an index or table of access points prior to content being made available for streaming. This is normally performed by a software manager or hardware manager of the time-based material stream, which may be referred to as a stream processor. The processing and memory demands associated with conventional stream processing techniques limit the number of streams and stream throughput that can be supported by a given content server. In some cases, specialized stream processors may be used to assist with stream processing.
As indicated, the various timestamps used to coordinate and synchronize the output of streaming content need to be accurate to prevent artifacts or other issues during playback. Conventional content streaming systems do not maintain the accuracy of all the timestamps used in a content stream through the transitions in and out of trick play modes, particularly, the decode timestamps (DTS) and presentation timestamp (PTS). As a result, the content stream may violate the encoding standard used and may have a degraded playback quality such as artifacts that appear at trick play transitions.
Throughout the drawings, identical reference numbers designate similar, but not necessarily identical, elements.